Skip to content

fix(server): downgrade expected connection teardown errors to debug#1369

Merged
drew merged 1 commit into
NVIDIA:mainfrom
russellb:fix/server-connection-teardown-logs
May 14, 2026
Merged

fix(server): downgrade expected connection teardown errors to debug#1369
drew merged 1 commit into
NVIDIA:mainfrom
russellb:fix/server-connection-teardown-logs

Conversation

@russellb
Copy link
Copy Markdown
Contributor

Summary

h2 connection errors during sandbox deletion are expected — the supervisor gRPC stream is torn down when the sandbox is removed. Downgrade these from ERROR/WARN to DEBUG so they don't pollute gateway logs during normal operations.

Related Issue

None — discovered during nftables migration e2e testing (#1335).

Changes

  • Add is_benign_connection_close() helper in lib.rs that matches expected h2/connection teardown error messages
  • Downgrade connection error logs from error! to debug! for both TLS and plaintext paths when the error matches a benign connection close
  • Downgrade ForwardTcp: inbound stream failed from warn! to debug! in grpc/sandbox.rs — stream teardown during sandbox deletion is expected

Testing

  • mise run pre-commit passes
  • Unit tests added/updated
  • E2E tests added/updated — verified gateway logs no longer show ERROR/WARN during normal sandbox lifecycle (create → exec → delete)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented May 13, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@drew
Copy link
Copy Markdown
Collaborator

drew commented May 14, 2026

/ok to test 1d988d9

@drew
Copy link
Copy Markdown
Collaborator

drew commented May 14, 2026

tests look good, but needs a rebase

h2 connection errors during sandbox deletion are expected — the
supervisor gRPC stream is torn down when the sandbox is removed.
Downgrade these from ERROR/WARN to DEBUG so they don't pollute
gateway logs during normal operations.

Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
@russellb russellb force-pushed the fix/server-connection-teardown-logs branch from 1d988d9 to 33449bb Compare May 14, 2026 15:34
@russellb
Copy link
Copy Markdown
Contributor Author

tests look good, but needs a rebase

thanks! rebase complete

@drew
Copy link
Copy Markdown
Collaborator

drew commented May 14, 2026

/ok test 33449bb

@drew drew merged commit 94025d8 into NVIDIA:main May 14, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants